A Study of Image Fusion Techniques in Remote Sensing

The amount and variety of remote sensing imagery of varying spatial resolution is continuously increasing and techniques for merging images of different spatial and spectral resolution became widely accepted in practice. This practice, known as data fusion, is designed to enhance the spatial resolution of multispectral images by merging a relatively coarse-resolution image with a higher resolution panchromatic image taken of the same geographic area. This study examines fused images and their ability to preserve the spectral and spatial integrity of the original image. The mathematical formulation of ten data fusion techniques is worked out in this paper. Included are colour transformations, wavelet techniques, gradient and Laplacian based techniques, contrast and morphological techniques, feature selection and simple averaging procedures. Most of theses techniques employ hierarchical image decomposition for fusion. IRS-1C and ASTER images are used for the experimental investigations. The panchromatic IRS-1C image has around 5m pixel size, the multispectral ASTER images are at a 15m resolution level. For the fusion experiments the three nadir looking ASTER bands in the visible and near infrared are chosen. The concept for evaluating the fusion methods is based on the idea to use a reduced resolution of the IRS-1C image data at 15m resolution and of the ASTER images at 45m resolution. This maintains the resolution ratio between IRS and ASTER and allows comparing the image fusion result at the 15m resolution level with the original ASTER images. This statistical comparison reveals differences between all considered fusion concepts. * Corresponding author. INTRODUCTION Every year the number of airborne and spaceborne data acquisition missions grows, producing more and more image data about the Earth’s surface. The imagery is recorded with varying resolution and merging images of different spatial and spectral resolution has become a widely applied procedure in remote sensing. Many fusion techniques have been proposed for fusing spectral with high spatial resolution image data in order to increase the spatial resolution of the multispectral images (Carper et al., 1990; Chavez et al., 1991; Kathleen and Philip, 1994, Wald, 2002). Data fusion as defined by Wald (2004) is a “formal framework in which are expressed the means and tools for the alliance of data originating from different sources. It aims at obtaining information of greater quality; the exact definition of 'greater quality' will depend upon the application. This approach is based upon the synergy offered by the various sources.” Focussed to the output of airborne and spaceborne sensors, i.e. recorded images, image fusion or image data fusion is concerned with methods for the fusion of images. The emphasis in this paper is put on images taken by different sensors with different spatial resolutions. The goal is either to visualize the original sets of images with improved expressiveness regarding its inherently available information, or to produce a new product of synthesized images with a better spatial resolution. The motivation of users of image fusion techniques often comprises both aims. Image fusion on the pixel level is sometimes also called pixel or signal fusion. If image fusion is related to the feature representation level of images it is also called feature fusion. Object or decision fusion deals with the high level representation of images by objects. The meaning of the terms feature and object in image processing (e.g. Gonzalez and Woods, 2002) and Remote Sensing (e.g. Wald, 2004) is still quite different from its use in Photogrammetry (e.g. Schenk, 2003) and Computer Vision (e.g Haralik and Shapiro, 1992). As a consequence the features in photogrammetry, in particular linear features extracted by edge detection schemes and areal features based e.g. on a texture segmentation scheme lead to an image description which is closer to an object description in image processing or pattern recognition than to a feature description. Image classification performed on a multispectral image may take in addition to the spectral data textural features and other feature image descriptions into account. At this point the difference between the different uses of terms is getting very obvious. Data fusion in its interrelationship with image analysis and GIS was reviewed in Baltsavias and Hahn (1999). Fusion of data recorded by different sensors has been put into context to data in GIS databases. Quite long is the list of problems of fusion related problems that have been worked out in the above quoted paper. The discrepancy between scene representations given by imagery and given by corresponding maps or GIS (vector) data sets links fusion concepts to topics of image analysis with the consequence that feature extraction, segmentation, classification, plays a important rule in particular for decision fusion. From an application point of view this addresses problems of automating mapping procedures and map update. In the following we will focus in image fusion techniques on the pixel level. The most well-known techniques, the IHS and PCA methods for colour composing, are already implemented in remote sensing packages; but some more advanced methods are methodologically or technically not yet mature. Altogether ten techniques will be mathematically described in the next section. All algorithms are implemented in MATLAB with the idea to create a MATLAB fusion toolbox. Experiments with IRS-1C and ASTER images are presented and discussed in Section 4. We finally conclude with a short summary and recommendations for future research. 2. IMAGE FUSION TECHNIQUES The number of proposed concepts for image fusion is growing which indicates ongoing research in this area. Technically, image data recorded by different sensors have to be merged or composed to generate a new representation. Alternatively data from one sensor are also subject of image fusion. Different multispectral channels are to be considered as different sources, as well as images taken at different times by the same sensor. The goal of all image fusion techniques is obtain information of greater quality which may consist of a more accurate description of the scene than any of the individual source images. This fused image should be more useful for human visual inspection or machine perception. The sensors used for image fusion need to be accurately co-aligned. Alternatively images from different sources may have to be registered or geocoded to the reference coordinate system. References for the algorithms worked out in the following are Anderson (1987); Burt (1992); Carper et al. (1990); Chavez et al. (1991); Kathleen and Philip (1994), Rockinger (1996) and Wald (2002). With respect to the conceptual approach, we distinguish the proposed techniques into eight classes of IHS, PCA, SWDT, Laplacian and FSD Pyramid, Contrast pyramid, Gradient pyramid, Selection and simple Averaging process. The main characteristics of these techniques are discussed in the context of its mathematical formulation. 1.1 Fusion based on Intensity-Hue-Saturation (IHS) method The Intensity-Hue-Saturation method (IHS) is one of the most popular fusion methods used in remote sensing. In this method, three multispectral bands R, G and B of low resolution are first transformed to the IHS colour space: